Motion picture restoration

This dissertation presents algorithms for restoring some of the major corruptions observed in archived film or video material. The two principal problems of impulsive distortion (Dirt and Sparkle or Blotches) and noise degradation are considered. There is also an algorithm for suppressing the inter-line jitter common in images decoded from noisy video signals. In the case of noise reduction and Blotch removal the thesis considers image sequences to be three dimensional signals involving evolution of features in time and space. This is necessary if any process presented is to show an improvement over standard two-dimensional techniques. It is important to recognize that consideration of image sequences must involve an appreciation of the problems incurred by the motion of objects in the scene. The most obvious implication is that due to motion, useful three dimensional processing does not necessarily proceed in a direction 'orthogonal' to the image frames. Therefore, attention is given to discussing motion estimation as it is used for image sequence processing. Some discussion is given to image sequence models and the 3D Autoregressive model is investigated. A multiresolution BM scheme is used for motion estimation throughout the major part of the thesis. Impulsive noise removal in image processing has been traditionally achieved by the use of median filter structures. A new three dimensional multilevel median structure is presented in this work with the additional use of a detector which limits the distortion caused by the filters . This technique is found to be extremely effective in practice and is an alternative to the traditional global median operation. The new median filter is shown to be superior to those previously presented with respect to the ability to reject the kind of distortion found in practice. A model based technique using the 3D AR model is also developed for detecting and removing Blotches. This technique achieves better fidelity at the expense of heavier computational load. Motion compensated 3D IIR and FIR Wiener filters are investigated with respect to their ability to reject noise in an image sequence. They are compared to several algorithms previously presented which are purely temporal in nature. The filters presented are found to be effective and compare favourably to the other algorithms. The 3D filtering process is superior to the purely temporal process as expected. The algorithm that is presented for suppressing inter-line jitter uses a 2D AR model to estimate and correct the relative displacements between the lines. The output image is much more satisfactory to the observer although in a severe case some drift of image features is to be expected. A suggestion for removing this drift is presented in the conclusions. There are several remaining problems in moving video. In particular, line scratches and picture shake/roll. Line scratches cannot be detected successfully by the detectors presented and so cannot be removed efficiently. Suppressing shake and roll involves compensating the entire frame for motion and there is a need to separate global from local motion. These difficulties provide ample opportunity for further research

A No-Reference Video Quality Predictor For Compressed Videos

A system and method to predict the perceptual quality of a compressed video by deploying a self-reference technique is disclosed. The method includes the steps of computing a frame difference image from the luminance component of at least one other alternate frame of an input test image. A blurred frame and a blurred frame difference image are then obtained by low-pass filtering of the input and frame difference images. A divisive normalization operator is applied on the four types of images independently and a generalized Gaussian distribution GGD fitted. Spatial features and temporal features are then extracted from the GGD. Absolute differences between the spatial and temporal features are computed and weighted based on motion in a given frame in the video. These features are pooled over different patches across the frames to obtain a final video quality score Q. The method shows superior results when compared to existing methods, while being computationally simple

Feature-based object modelling for visual surveillance

This paper introduces a new feature-based technique for im-plicitly modelling objects in visual surveillance. Previous work has generally employed background subtraction and other image or motion based object segmentation schemes for the ſrst step in identifying objects worthy of attention. Given that background subtraction is a notoriously noisy pro-cess, this paper investigates an alternative strategy by instead employing feature (SIFT [1]) clustering to characterise ob-jects. The segmentation step is therefore performed on the sparse feature space instead of the image data itself. The paper also presents an application employing this idea for automatic detection of illegal dumping from CCTV footage. The Viterbi algorithm then allows robust tracking [2] of ob-jects generated from the spatial clustering of these sparse foreground feature maps. Index Terms — visual surveillance, SIFT, background modelling, foreground estimatio